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LISTING OF CLAIMS: 

^\ ; 

Claim 1 (Currently Amended): An information input processing computer system for mapping 
gestures to keys of a virtual keyboard, the system comprising one or several cameras, one or 
more memories with CPU connected to the cameras, and processes running in the CPU that 
aasooiatoq associate gesture movements with typing and produce gesture associated textual 
output, wherein said processes capture gesture images, classify each types of gesture image into 
a respective one of a plurality of classes depending on the type of gesture, and associate each of 
the classes with one of the koyo of the virtual keyboard or more possible kevs. assign a 
probability to each of said possible keys, and integrate the probabilities assigned to the possible 
kevs to identify a word for a sequence of gestures . 

Claim 2 (Original): The gesture-key mapping system as in claim 1, where a feedback is 
provided to the user on what kind of keys are associated with the user's gestures. 



Claim 3 (Original): The gesture-key mapping system as in claim 2, where the feedback is 
provided using one or more of the following: displaying keys on a display, playing sounds labels 
for keys, displaying image indicators on a display, playing special sound indicators, projecting 
the keyboard to any surface, and displaying picture of the keyboard with user's hands. 
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Claim 4 (Currently Amended): An information input processing, gesture-key mapping computer 
system, the system comprising one or several cameras, one or more memories with CPU 
connected to the cameras, and processes running in the CPU that assooiatoD associate gesture 
movements with typing and produce gesture associated textual output, where the gesture-key 
processing is provided using the following modules: 

a) a gesture capturing module that captures gestures through camera sensors; 

b) a gesture classificator module that classifies each type of gesture into a 
respective one of a plurality of classes ef depending on the type of gesture movements; 




c) an associator module for associating gesture classes or sequence of gesture 
classes with one or s e veral moot probable more possible keys and assigns a probability to each of 
said possible keys: and 

d) an integrator module that intograto sequenc e of oandidoto of most probabl e 



keys into uniquo output key ooquonoo integrates the probabilities assigned to the possible keys to 
identify a word for a sequence of gestures . 



Claim 5 (Original): The system as in claim 4, where the integrator module includes one or more 
of the following: 

a) language module component that estimate probabilities of word strings corresponding to key 
candidate sequences; 

b) character frequency module that estimate probabilities of character strings corresponding key 
candidate sequences; 

c) confusable matrix mat estimate how often correct gesture classes are confusable with another 
gesture classes; 
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d) gesture classes probability model that estimate probability of observing a string of gesture 
classes given a sequence of gesture frames; 

e) computation of a probability of production a sequence of keys given a string of 
gesture frames; 

f) generation of a lattice of sequences of keys given sequence of gesture frames; 

g) finding the most probable sequence of keys from the lattice of key candidate strings. 

Claim 6 (Original): A system according to Claim 5, wherein each sequence of keys receives a 
probability score, and the sequences of keys that receive low scores are removed and are not 
continuing when new candidates for keys arrive. 

Claim 7 (Original): A system according to Claim 1, further comprising a gesture correlator 
module to allow to adjust automatically an invisible keyboard to hand positions. 

Claim 8 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gestures without the presence of a keyboard and the gestures are associated with the 
most probable keys that would be typed if a keyboard were presented, said method including the 
steps of using a computer system to map gestures to keys of a virtual keyboard, including the 
steps of r unnin g processes on the computer to capture gesture images, to classify each type of 
gesture image into a respective one of a plurality of classes depending on the type of gesture, and 
to associate each of the classes with one of tho koys of tho virtual keyboard or more possible 
keys, assign a probability to ea ch of said possible kevs. and integrate the probabilities assigned to 
the possible keys to identify a word for a sequence of gestures . 
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Claim 9 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gestures without the presence of a keyboard and the gestures are associated with the 
most probable keys mat would be typed if a keyboard were presented, said method including the 
step of using a computer system to map gestures to keys of a virtual keyboard, including the step 
of running processes on the computer to capture gesture images, to classify each type of gesture 
image into a respective one of a plurality of classes depending on the type of gesture, and to 
associate each of the classes with one of tho keys of tho virtual koyboard or more possible keys, 
assign a probability to each of said possible keys, and integrate the probabilities assigned to the 
possible keys to identify a word for a sequence of gestures, and wherein the probability is 
computed using HMM. 



Claim 10 (Currently Amended): A method of typing using a virtual keyboard having a multitude 
of virtual keys, comprising the steps: 

making typing gestures without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output 
including the step of running processes on a computer to capture gesture images, classify each 
type of gesture image into a respective one of a plurality of classes depending on the type of 
gesture, and to associate each of the classes with one of tho virtual k e ys of th e virtual keyboa rd 
or more possible keys* assign a probability to each of said possible keys, and integrate the 
probabilities assigned to the possible keys to identify a word for a sequence of gestures . 



5 G:\tbmU05\l33l7\amendtt33l7.am4.doc 

PAGE 10/17 1 RCVDAT 4/16/2004 10:49:04 AM [Eastern Daylight Time] * SVR:USPTG-EFXRM/t) ' DNlS:8729306 1 CSID : 5 1 67424366 * DURATION (mnws):04-22 



ftPR- 16-2004 11:02 SCULLY SCOTT 5167424366 P. 11/17 

Claim 1 1 (Original): A method according to Claim 1 0, wherein the typing gestures are made by 
a person, and further comprising the steps of providing feedback to the person on texture output 
associated with the gestures. 

Claim 12 (Original): A method according to Claim 11, wherein the step of providing feedback 
includes the step of displaying an image of typing keys associated with the gestures. 

/ / A Claim 13 (Original): A method according to Claim 10, further comprising the step of generating 
{J^ an image of a keyboard; and wherein the step of making typing gestures includes the step of 
. making typing gestures relative to said image es as if said image were a real keyboard. 

Claim 14 (Currently Amended): A method of typing using a virtual keyboard, comprising the 
steps: 

making typing gestures without any real keyboard; 
sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output: and 
wherein the producing slep steps includes the step of classifying each type of gesture into a 
respective one of a plurality of classes depending on the type of gestur e, associating each of said 
classes with one or more possible keys, assigning a probability to each of said possible keys, and 
integrating the probabi lities assigned to the possible kevs to identifying a word for a response of 
gestures . 
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Claim 15 (Original): A method according to Claim 14, wherein the producing step further 
includes the step of associating gesture classes with individual typing keys. 

Claim 16 (Previously Presented): A method according to Claim 14, further comprising 
providing training data in words or sentences with certain timing data. 

Claim 1 7 (Currently Amended): A typing system using a virtual keyboard, comprising 
means for sensing typing gestures made without any real keyboard; and 

means for producing, from the sensed typing gestures, gesture associated textual 
output said producing means including a computer and processes running on the computer to 
capture gesture images, to classify each type of gesture image into a respective one of a plurality 
of classes depending on the type of gesture, and associate each of the classes with one of the 
keys of the virtual key board or more possible keys, assign a probability to each of said possible 
keys, and integrate the probabilities assigned to the possible, keys to identify a word for a 
sequence of postures. 

Claim 18 (Original): A system according to Claim 17, wherein the typing gestures are made by a 
person, and further comprising means for providing feedback to the person on texture output 
associated with the gestures. 

Claim 19 (Original): A system according to Claim 18, wherein the means for providing 
feedback includes means for displaying an image of typing keys associated with the gestures. 
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Claim 20 (Original): A system according to Claim 17, further comprising means for generating 
an image of a keyboard; and wherein the sensing means includes means for sensing typing 
gestures made relative to said image ©s as if said image were a real keyboard. 

Claim 21 (Currently Amended): A typing system using a virtual keyboard, comprising 



output; and 

wherein the producing means includes means for classifying each type of gesture 
into a respective one of a plurality of classes depending on the type of gestu re^assptiating each 
of said classes with one or more possible keys, assigning a probability to each of said possible 
keys, and integrating the probabilities assigned to the possible keys to identifying a word for a 
response of gestures . 

Claim 22 (Original): A system according to Claim 21, wherein the producing means further 
includes means for associating gesture classes with individual typing keys. 



means for sensing typing gestures made without any real keyboard; and 



means for producing, from the sensed typing gestures, gesture associated textual 
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